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EXECUTIVE SUMMARY 

Display format and highlight validity studies (Tullis, 1984; Fisher 
shown that these factors affect visual display search performance; however, these studies 
been conducted on small, artificial displays of alphanumeric stimuli. A study maculating these 
variables was conducted using realistic, complex Space Shuttle information d P y • 

(display type: Orbit Maneuver Execute, Relative Navigation) x 2 (d.splay format current ^ 
reformatted [following human-computer interface design principles]) ) x : 3 } ^ f 

valid, invalid, no-highlight) within-subjects analysis of variance found significant main effects o 
these variables on search time. Search times were faster for items in reformatted displays 
compared to current displays. Responses to valid applications of h.ghligh1 : were 
faster than responses to non-highlighted or invalidly-h.ghl.ghted applications. The ^gnificant 
format by highlight validity interaction showed that there was little difference in resp 
both currentand reformatted displays when the highlight was v a |,d| y applied; however und 
the no-highlight and the invalid highlight conditions, search times were faster with reforma 

displays^ analysis studied the interaction between display format, highlight validity, and 

several' iSST^M The 2 (display format: current, reformatted) x 2 0*g*0£*. 
valid invalid) x 4 (highlight method: brightness, color, flashing, reverse video) within-subj 

analysis of variance did not reveal a main effect of highlight method nredicted bv 

In addition, observed display search times were compared to search times predicted oy 
Tullis’ Display Analysis Program (1986). Significant correlations between predicted and ob- 
served search times were found, although the relationship was highest with no^highhghted 
displays (r= .75) and was less predictive with valid (r= - .52) and invalid ([- .31) app 
highlight. Issues discussed include the benefits of highlighting and reformatting displays to 
enhance search, and the necessity to consider both highlight validity and format characteristics 
in tandem when predicting user search performance. 
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1.0 INTRODUCTION 


Much research has examined factors which optimize operator search and selection of 
information from alphanumeric computer displays. Two particular issues have dominated t is 
research in recent years: the effect of format characteristics on search performance has been 
the focus of work conducted by Tullis (1981 , 1983, 1984), and the demonstration of search 
performance under different levels of highlighting validity has been addressed by Fisher, oury, 
Tengs, and Duffy (1989), Fisher and Tan (1989), Tan and Fisher (1987), and Warner, Juola, 
and Koshino (in press). In Tullis’ dissertation (1984), four format characteristics (overall display 
density, local density, number of groups, and average visual angle of groups) were significant 
predictors of the time needed to search for a single item of information in a display. Fisher ana 
Tan (1989) found differences in highlight benefit under different levels of highlight validity. when 
highlight validity was less than 50%, performance with highlighted displays did not differ from 
performance with displays on which no highlight was present. 

In a comparison of format and highlight as predictors of search performance, format can 
be viewed as the more basic of the two variables. The format of an information display is 
generally fixed: no changes are made to the overall layout of the information after the display 
has been implemented. It is at this point that highlight may be applied in order tojncrease 
target search and detection. In this sense, highlight may become a subsequent, quick-fix' 
solution for poor display design. 

However, some research has sought to identify format characteristics that influence 
search, thereby circumventing the need to apply highlight techniques. Characteristics included 
in this group are such display format variables as item alignment (Streveler & Wasserman, 
1984), and display loading (defined as the percentage of active area on a screen) and grouping 

(Danchak, 1976). J ... ^ ... H 

Recently, much of the interest in format has been concentrated on the variables studied 

by Tullis (1984). Tullis initially proposed six display format characteristics as possible predictors 

of search time: .. .. .. 

1) overall density- percentage of the total number of character spaces available on the 

display devoted to actual display characters; 

2) local density- a weighted measure of how close characters were to each other wit in 

a 5-degree visual angle; 

3) number of groups- a simple count of the number of groups on the display, 

4) average visual angle of groups- a weighted average (based on the number of charac- 
ters in a group) of the angles subtended by the groups of information on the display; 

5) number of items- a simple count of the number of character items present on the 

6) layout complexity- a measure of information alignment (vertical and horizontal 
complexity measures, based on information theory, are summed to provide a total measure of 

layout complexity). , „ , 

Tullis’ model was derived via a “regression by leaps and bounds multiple regression, 

which provides the best-fitting regression model for each number of predictor variables. Tullis 
found that each of these search time regression models included at least one of the Jwo group- 
ing variables (i.e., number of groups or average visual angle of groups). The overall 6-predictor 
model accounted for 50.8% of the variance in his subjects’ response times. 

Tullis selected the 4-variable model (containing overall density, local density, number ot 
groups, and size of groups), accounting for 48.8% of the variance, as the best model, since the 
variables in the 5-variable (adding the number of items variable) and 6-variable mode's did not 
add significantly to the prediction. This regression model forms the basis for Tullis Display 
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the opportunity to determine how well the Tullis 

model ^edicts actual search time for more complex displays of * 

response times from the present study and the predicted vahiee ^ search performance, 

measure of the extent to which the Tullis regression model predicB atfua I se^ch penomn^ 

It must be noted that a difference between Tullis-predicted and observed r p tim __ 

L re a^"t986). Tullis has addressed this possibility: The predict^ search un^ . . . 

although they [predicted response times] generally have correlated v^with mT" ideally 
the literature* the absolute values of the predictions have commonly been wrong . . . pe y. 
thft nreriirted search times have often been too low” (pp. 11-12). 

p For eS^te a significant correlation would imply ttetpooriy-des^ dis^ayshad 

longer predicted search times, as found by bfilh the Tullis ^ 

this rase the Tullis assertion of predicted search times as relative would be s pp 

has focused on highlighting as a potential me^ °r e^ancing w ual 

display search performance. Highlighting is an obvious choice as an M^toation 

search of a display for two reasons: (a) color coding, as one exampte ofag 9 PP 

has been demonstrated to be effective when searching for a target in an anay (Christ, 19 ), 
and (b) the widespread application of highlighting techniques in visua displays with little empir • 
nal research Drompts an investigation of the implications of highlight use. 

SShat highlight will be applied to a non-target display item is one such 

where the target was highlighted when compared to performance when a detractor was high 

'' 9hted Both the validity and the format studies mentioned here have been conductedondis- 
plays containing small sets of stimuli. The present study investigated the impart ol highlighting 

°" diSP l^ 

tested in order to provide a greater range of displays. 
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2.0 METHOD 


2.1 Subjects 

Twelve Lockheed Engineering and Sciences Company (LESC) employees voluntarily 
participated in the experiment. All subjects had normal or corrected-to-normal vision, and were 
unfamiliar with the experimental displays. 


2.2 Stimulus Displays 

Two current Space Shuttle displays, Orbit Maneuver Execute and Relative Navigation, 
were presented on an IBM PC/XT with a 13-inch color monitor. These alphanumeric displays 
were roughly similar in appearance, but they contained information particular to either the Orbit 
Maneuver Execute or the Relative Navigation task. Regardless of condition, the information 
occupied almost the entire display area of the monitor. The displays were identical to those 
used in Burns, Warren, and Rudisill (1985). 

Each display type was presented under two kinds of format: current and reformatted. 
Format changes to improve the current navigation and orbit displays followed good human 
factors practices (e.g., grouping, alignment of numeric data values) and are enumerated in the 
Burns et al. paper. Examples of the current and reformatted versions of the navigation and orbit 
displays are shown in Figures 1 and 2. 

Dan Bricklin’s Demo II program was used to add highlighting to the displays and to 
present the stimuli during the study. 


2.3 Experimental Design 

The experiment followed a 2 x 2 x 3 within-subjects factorial design. The displays 
presented to each subject varied by combinations 
of display type (Orbit Maneuver Execute or Relative Navigation), display 
format (current or reformatted), and highlighting validity (valid, invalid, or no-highlight). Subjects 
were presented with two replications of each of the different display combinations. Each of the 
display type by format display combinations was presented twenty 
times. To avoid order effects, four random sequences of display presentations were used. 

Highlight was present on 80% of the displays, equally divided into valid and invalid use 
of highlight. The highlighted item was either the target item (valid application of highlight) or a 
randomly-selected distractor display item (invalid application of highlight). The remaining 
displays (20%) had no highlight and served as a control condition for the use of highlight. When 
present, the highlight was one of the following techniques: brightness, color (blue), flashing, and 
reverse video. With the exception of the displays in the blue-highlighted condition, all displays 

were monochrome. ... 

As in the Tullis (1984) experiment, the dependent measure in this experiment was time 
needed to supply an answer to a randomly-selected question concerning information contained 
in the display. 
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Figure 1. Examples of the current (top) and reformatted (bottom) Relative Navigation experimental 
displays. r ; . r .- :r _ 


2.4 Procedure 

Subjects performed a computer-generated search and identification task. In each trial, a 
question concerning one item of Space Shuttle display information was presented (e.g. "What is 
the value for Cl ?", "What is the value for node?”). A set of twenty questions was used. The 
subsequent display contained the information needed to answer the question. Subjects were 
instructed to signal when they had found the information by pressing the Return key. Response 
time was defined as the time between information display onset and the press of the Return 
key. At this point, a response screen appeared and the subject typed the answer value for that 
question. Subjects received a short break after the first 40 displays. The experiment was self- 
paced; subjects generally completed the experiment within one hour. 
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Figure 2. Examples of the current (top) and reformatted (bottom) Orbit Maneuver Execute experimental 
displays. 
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3.0 RESULTS 


3.1 Analysis of the Factorial Design 

A 2 x 2 x 3 (Display Type x Display Format x Highlight Validity) analysis of variance 
(ANOVA) revealed that significant display type differences were present, F (1 , 132) = 12.09, p < 
001 Subjects took longer to respond to the navigation displays than to the orbit maneuver 
displays (mean response time = 17.51 and 13.29 seconds, respectively). The main effect of 
display format was significant, E (1 , 1 32) = 7.22, p < .01 . Response time to the reformatted 
displays was significantly shorter (mean = 13.77 seconds) than response time to the curren 
displays (mean = 17.03 seconds). There was a significant difference in the time needed to 
respond under the three highlight applications, E (2, 132) = 22.42, p < .0001 . As expected, a 
Student-Newman-Keuls comparison of means showed that mean response time to valid appli- 
cations of highlight (9.66 seconds) was significantly shorter than mean response time to dis- 
plays in which no highlighting was present (18.00 seconds) and to invalid applications of high- 
light (18.54 seconds) at the p < .05 level. The difference in response times to no-highlighting 
displays and displays with invalid application of highlight was not significant. 

Display type did not enter into any of the higher-order significant interactions. Among 
these higher-order interactions, only the format by highlight validity interaction was significant, E 
(2, 132) = 3.62, p < .05. For the current displays, a Scheffe test showed a significant difference 
between valid highlight (mean = 9.04 seconds) and the other highlight validity conditions (p < 
.05). There was no significant difference between no-highlight (mean = 21 .20 seconds) and 
invalid highlight (mean = 20.86 seconds) for current displays (p < .05). For the reformatted 
displays, there was a significant difference between the valid (mean = 1 0.28 seconds) and 
invalid (mean = 16.22 seconds) conditions (p < ,05). The no-highlight condition (mean = 14.81 
seconds) did not significantly differ from the other conditions (p < .05). The interaction is illus- 
trated in Figure 3. 


3.2 Factorial Analysis of the Highlighting Methods 

In this section, the difference in response times to several highlight methods was investigated. 

In order to accomplish this analysis, the no-highlight trials were dropped from the analysis. The 
data was collapsed across display type for two reasons: (a) the effect of highlight under the 
different display types was not of interest, and (b) display type did not interact with either the 
display format or the highlight validity variables in the above factorial. The analysis followed a 2 
(display format: current, reformatted) x 2 (highlight validity: valid, invalid) x 4 (highlight method. 

brightness, color, flashing, reverse video) within-subject design. 

The 2x2x4 ANOVA revealed that the main effect of highlight validity was significant, E 
(1 , 176) = 63.53, p < .0001 , as in the above analysis. Response time to valid highlight (mean = 
9.48 seconds) was significantly shorter than response time to invalid highlight (mean = 1 8.46 
seconds). Response time to reformatted displays (mean = 13.09 seconds) was shorter than 
response time to current displays (mean = 1 4.85 seconds), but the main effect of display format 
was not significant, E (1 . 1 76) = 2.46, p < .1 2. The main effect of highlight method was not 
significant, E (3. 176) = 1 .65, p < .18. Response times to the highlight methods were as f°JJ 0WS * 
color (mean = 12.23 seconds), flashing (mean = 13.46 seconds), brightness (mean = 14.6 
seconds), and reverse video (mean = 15.59 seconds). 
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Among the higher-order interactions, only the display format by highlight validity interac 
tion was significant, E (1 , 176) = 6.93, c < .01 , similar to the previous analysis. 
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Figure 3. Mean response time as a function of highlight validity and display format. 

3.3 Observed and Predicted Search Time Performance 

Of interest in this section was the correlation between the observed dis P^y|®^ ch times 
and the Tullis-predicted search times. Using Tullis’ Display Analysis Program (1986), the 
experimental displays were evaluated on overall density, local density, number o groups, 
average visual angle of groups, number of items, and format complexity. The program also 
provided measures of predicted search time for each experimental display. 

P The correlation between predicted and observed search time varied, depending on the 
validity or presence of highlight. The correlation between predicted and observed search times 
for non-highlighted displays was high and significant, r = .75. * < .0001. The correla hon was 
lower for displays with valid (r = - .52) and invalid (r = .31) applications of highlight, although 

both sets of correlations were significant at the p < .05 level. .. 

Although the association between observed and predicted search times was strong, the 
predicted search time values and the observed values differed in magnitude. ® 
predicted mean search time was 3.4 seconds. This predicted time was similar *1* uT 

highlight validity conditions, since the Tullis model calculated predicted searc seconc i s \ 

U ej n n hiahliaht as a factor The overall observed search time was much longer (15.4 ). 

4 stated ^Section 3.1, the mean observed search times for the valid, no-highlight, and invalid 
highlight conditions were 9.66, 18.00, and 18.54 seconds, respectively. 
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4.0 DISCUSSION 


The present study strengthens the generalization of previous display format an ^ high- 
light validity findings taken from studies using more artificial, less complex informati on disp lays. 

The important findings centered around the highlight validity and the display fornr \ 
manipulations. In this study, significantly shorter search times resulted from both an appkcatton 
of highlight on the appropriate item of information and a reformatting of the current display . 

Performance was best when highlight was validly applied to a target item, regar ess 
the display format. Despite less than optimal display format in some conditions items were 
found quickly simply because the subject’s attention was drawn to the Ion® highlighted item on 
the display. This highlight benefit was not dependent on the method of highlight. 

Format became an issue when the highlight was either not present or was invalid y 
applied. In these situations, the user’s task involved search through the display. In both ot 

these conditions, the search was made faster by display redesign. . . roctinn 

The impact of display format on search performance in this study offered two interesting 
findings. A reformatting of the current displays had a significant influence on ^e speed wrth 
which subjects could locate items on a display, with subjects searching the reformatted displays 
more efficiently than the current displays. A search performance benefit of highlight was y 
found under the current displays: the addition of highlight did not aid user search more than a 
re-grouping of display items. The reformatting was based upon general formatting principles 

such as alignment and grouping. . „ ... 

In addition, this study demonstrated that Tullis’ Display Analysis Program predicts actual 

search performance with complex, as well as simple, displays. This correlation was highest with 
the non-highlighted displays, and decreased with the application of display highlight. This 
finding suggests that, in the absence of other factors affecting search (e.g., highlighting), these 
format characteristics are strong predictors of performance. However, the lower association 
under highlighted conditions is evidence that format characteristics should be considered to- 
gether with other search factors when attempting to optimize the prediction of user search 
performance. 
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